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This paper evaluates the effects of two distinct 
modes of feedback to students. These two modes comprise: (1) 
predicted final course grade, and (2) current standing in class. The 
experiment was conducted with a final year class of 56 students " 
attending a required half-year course in Educational Measurement. The 
class was divided into four treatment groups (2x2 factorial design) 
and different feedback of results was mailed to each group for the 
four objective class tests written before final exam. Two FORTRAN 
computer programs were used to determine class standing, current 
grades and predicted final course grades. The predicted final exam 
and course grades were obtained by applying a linear least-squares 
line of best fit to the test data and extrapolating to future tests 
and to the final exam. The class standing was obtained by using a 
FORTRAN sorting program. Analysis of variance was performed on the 
final exam grades, and no significant differences were found in final 
exam performance or in satisfaction with feedback received 
(Chi-squared test) , although results slightly favored the feedback 
groups. The computer program was found to adequately predict the 
final exam grades, i.e., the linear regression was significant. 
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EVAL UATION OF A COMPUTER-BASED FEEDBACK SYSTEM 



ABSTRACT 

This paper ovaluates the effects of two distinct modes of feedback to 
students. These two modes comprise: (1) predicted final course grade, 
(2) current standing in class. 

The experiment was conducted with a final year class of fifty-six studants 
attending a required half year course in Educational Measurement. The class 
was divided into four treatment groups (2x2 factorial design) and different 
feedback of resu' ts was mailed to each group for the four objective class 
tests written before final exam. ^ 

Two FORTRAN computer programs were used to determine class standing, 
current grades and predicted final course grades. The predicted final exam 
and course grades were obtained by applying a linear least-squares line of 
best fit to the test data and extrapolating to future tests and to the final 
exam. The final course grade is simply a summed weighted composite of the 
class tests and 'inal exam. The class standing was obtained by using a FORTRAN 
sorting program. 

Analysis of Vc^riance was performed on the final exam grades and no sig- 
nificant differences were found in final exam performance or in satisfaction 
with feedback received (Chi-squared test), although results slightly favoured 
the feedback groups. The computer program was found to adequately predict 
the final exam grades , i.e., the linear regression was significant. 



EVALUATION OF A COMPUTER-ABASED FEEDBACK SYSTEM 



Introduction 

Ley (1) has summarized the advantages of short, frequent quizzes as leading 
to increased motivation, elimination of cremming, provision of continuous 
knowledge of student progress and removal of the fear of final examinations. 
He has developed a FORTRAN computer program which can be uspd for calculating 
student grades as well as for predicting the final exam and course grades. 
A program has been written by the authors to rank order the students after 
each test. 

The purpose of this experiment was to evaluate the accuracy of the program 
as a predictor of final exam grades and to determine whether or not different 
modes of feedback from class tests resulted in significantly different final 
exam performance. The students' satis-'action with the different modes of feed- 
back was also tested. 

Desig n 

Fifty-six f .nal year Education students attending a li^ unit course in 
Measurement at the University of British Columbia were used as subjects in the 
experiment. The students were randomly assigned to one of four treatment condi- 
tions as shown in Figure 1. The unequal cell sizes were caused by students 
withdrawing from the course. Feedback Condition A will be referred to as 
Morm-Referenced , and consisted of information regarding current class standing. 
Feedback condition B, referred to as Crite rion-Referenced , consists of info-niation 

1. Ley, James B. "Grades, Quizzes, Motivation, and Computers", IEEE 
Transactions on Education, Vc- . E-13, No. 3, September, 1970. 



about predicted final course grade. The feedback given to each group Is 
summarized in Table 1 . 



B 

(Criterion) 
yes no 



n = 15 


n = 14 


n = 14 


n =»13 



T^ble 1 •• Summary of Feedback Conditions 



Feedback Mode : 


Information given to students 


A 


B 


Average % of 
correct responses 


Class standing 
(which quarter) 


Predicted final 
course grade 


yes 


yes 


X 


X 


X 


yes 


no 


X 


X 




10 


yes 


X 




X 


no 


no ' 


X 







In order to minimize "leakage" between groups, the results were mailed 
to the students' homes after each quiz. Questions by some students about 
differences in the feedback were answered discretely by telling the students 
that the instructor wished to determine the feedback most satisfactory to them. 

Four tests and a final examination were developed and administered in 
class by Prof. R. Conry, the instructor for the course. The tests were 
approximately one half-hour long and one hour was allotted to the final 
examination. Th2 inter-test correlations are given in Table 2. 



Table 2 - Test and Exam Correlations 





Tl 


' T2 


T3 


T4 


F. Exam 


Tl 


1.000 


.423 


.270 


.437 


.412 


12 ■ 




1.000 


.169 


.188 


.296 


T3 






1.000 


.347 


.31^ 


T4 








1.000 


.231 


F. Exam 










1.000 



Com puter Progra m 

The computer programs used to obtain the feedback are given below. 
It should be notf d that only certain parts of the overall output were 
revealed to each group. 



C CALC'^lATIC^J AND PREniCTirN OF COl/RSr GKAOES 

^01 WRIT (/S,33) 

Zl^ ^^^JOn^ "^dHltPOXt^CLASS MARKS'///) 

^ . . . ^ 

c 

C ' 

C THF T.-^ItT CATA IS READ INTO THE COMPUTFR 

C 



» 1 1 



1^ ; 
} ■ 



ERLC 



.^Of 1'^^ READ ( , 1 1 EMn = 200) ( ANAME 1 1) . I =1 ,?0 ) .TfyMUVGl j ) , J= 1 , T9T 
^•^^ 1 F0RMAT(?0AUF3.0,19(F3*0)) 

C 

C THE CAl.C^)LATI^^j OF THE AVERAGE GRAPE 

C 

_nSUf'_=n.p 

p':? " 00 10 r=i\io" 

IF(G( I ) )K, .^OtlO 
GO T( 15 

• 15 AVE = G? MVK ^ \ 



r 



J; S'E WHFTfJER THE FINAL F-XAM GRAPE IS PFCOROFO 

C 

C LIKTA'^ I.FA^.r f OllAP FS P^E ^ I CT ION _ 0 F thp fimal EXAM GRAPE. 
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SUM 3-. \!JM ^4 ! f 

f rx A' .>»»-:^i 



r 

r 



rrt. ATM .-a;/! .A^FP VAM 



- riU'MM/]M Avn^.u;r rr'i,f/,,f ..kaocs :t,r^.i, :^iiMA- 

? t / / ) 



I ^IHb I "f-^l AVERAGE OF A GRAOES = 65.0 

ESTIN^ATEO FINAL f-XAM rHAD E = 62.0 ESTIMATED FINAL COURSE GRADE = 63.2 



1 ^BBHiP 2 ■">?2 AVERAGE OF 4 GRADES = 62.8 
ESTIMATEn FINAL f <AM GRAHE = 55,0 'ESTIMATED FINAL COURSE GRa6~E ^: "58.1 



2 Ib^h 'AVEkAGE CF 4 GRAQES = 7173 
ESTIMATED FINAL F'<AM GRADF = 72.0 ESTIMATED FINAL COURSE GRADE 72.5 



liHBi 2 14 AVERAGE OF 4 GRADES = 56.8 

J*^Tr^'^T§'-' LIJl^L !'^V\?'''iPf:.Z....'l2.*_9 I^IL'^'^^F'^ FI^JAL course grade = 58.7 



^iKttttt^ 2 f051 '\VEPAGF OF '-t GRAOFS = 5^.? 

ESTIM-ATEH FINAL PXAM ' '^AnF = 78.0 ESTIMATED FU'AL COUR GRADE = 69.3 



i^PP ■ - 2 } '2' /'''f l ''"GF OF ■ OR A'nt 5""'^"!":.""'^ 

rSTIMAFFi' FINAL' ■■' 'AM ' T^'f- = 72,0 F^TI'iATFO FlfAl. CPUR ".E GRADE = 74.6 



STOP n _ 

EX' CUTinN TERMIfiAT. n 



IS IG 



f rRTRAN" I V G COMPILER 



17714:6 7 PA G 6~0C 



C PPOGPAM T'l P.AfiK IIPDF.R STUOENTS 
C 

C DATA CAR IS ARE 1. FORMAT CARD OF ^:ORM t20Al,F3.0) 

C 2 . NUMBER OF STUDENTSt CO L . 1-3) 

3. 'data CCNSISTING of "id rODECCOL. 1-20) AND MARKS 

c 

001 DIME 'l^IOM F'-'T(20) ,NAME( 2CC,20) ,GRADF(200) ,TNAM(20) 

002 REACC^,! JF'iT 
: 003 1 FORW i": (20A4) 

004 _______ REAC(S 2)N 

00 5. 2 F0PM/iT(i3) 

006 no 3 •^=1,N 

007 3 REAC<t, FMT) (NA!1E(M, J), J=l,20),GRADi (M) 
ore N1=N-1 

• cc<; DO II 1 = 1, Ml 

• oip LLrlll 

oil on 11 j=ii,N 

'012 IF( GRADE ( I)-GRADE(J)) 14,11,11 

<013 14 TEMP=^GRA0E{ I ) 

014 GRADi ( I )=G'UDF( J) 

• 015 ' GRAD ( J) = T<-;'',P 

OIJ^. 00 2 l. = l 

017 " TMAM L)=^:A'■"Z( I ,L) " 

■Ua NAME' I ,L >=NAMFl J,L) 

t'l*? ^0 NAME' J,L)=TNAf (L) 

■ 020 1 1 CGNf: ^ )E 

')2 I WRITt ( i, 4 ) 

^2 ? _ _ 4 _FC^^AT ( 1 HI , 10> . 'RANK CRDER'/// ) _ 

,-,2 3 ^ " no 30 '=!;>.'" - - - - - 

'.) WRIT'! (' , M (MA'11 (fM ) , [ = 1, 20) ,GPAOF( Ml ,M 
■T-""> 7 FPRMATdH ,?:)Ai ,F'-^.0,ir)) 

■■'2'^ STtiP 
-)?7 END 

IHTM. TlO^iTRV R^i OUIfOi A T ( ',6C.A "BYtF--; ""' - ■ ■ 

(.n^r iLf r IMC = . <; rnfi is 
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Analysis and Res ults 

All analyses were carried out at the UBC Computing Centre using an IBM 
360/67 Computer. 

The programs used in the analyses are summarized in Table 3. 



Table 3 - Computer Programs 



Analysis 


Program 


* The BMD programs are from the UCLA 


Analysis of Variance 


BMDX64* 


Biomedical Package of Statistical 


Regression Analysis 


BMDOZR^ 


routines . 


Chi - Square 


IWTAB** 


** MVTAB is a UBC computing centre program 



Analysis of variance on the final exam grades was performed (Table 4b,» 
to test or differences in performance between the cell means (Table 4a) of 
the four groups. 

Table 4a - Cell Means for Final Exam 
yes ^ no 



yes 


! 67.5 


72 


3 


no 


65.1 


65 


9 



Table 4b - ANOVA 



Source of Variation 


df 


ms 


■ F 


P < 


A 


1 


271.86 


1.81 


.17 


l\ 


1 


109.64 


.73 


.40 


A X B 


1 


53.12 


.35 


.56 


ERi^OI^ 


52 


150.24 







stepwise regression analysis was performed using the final exam grade 
as the predicted variable arid the test grades as predictor variables 
(Table 5a). The significance of the linear regression used in the predict- 
ing program was tested. (Table 5b) 

Table 5a - Stepwise Regression 



step 
No. 


Variable 
Entered 


Multiple 
R 


I 

r2 


Increase in 
P2 


F value 
to enter 


No. of ind. 
variables included 


1 


Test l(tl) 


.423 


.179 


.179 


11.75 


1 


2 


Test 2(T2) 


.468 


.219 


.041 


2.75 


2 


3 


Test 3(T3) 


.565 


.319 


.099 


7.59 


3 


4 


Test 4(T4) 


.607 


.368 


1 .049 

1 


3.99 


4 



F. Exam - 11 J + .195 (Tl) + .027 (T2) + .233 (T3) + .155 (T4) 



Table 5b - Anova "or Regression 



I ' 'I I 

Source )f Variation df ' ms F 

I i : 

i Regression ! 4 i 549.27 7.43 



P< 
.01 



j Residual 

i 



\ 51 



7J.94 



A Chi - Square analyfJs was performed on the bivariate frequency cables 
of feedback condition vs. satisfactioi with feedback received (yes or no). 
The analysis was carried out on an overall basis (Table 6a) and separately 
fo"^ the two feedback modes (6b. 6c). 
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Feedback 
mode 



Table 6a ~ B'* van* ate Table of Feedback vs So^ '-^f action 
Satisfaction 





No 


Yes 




A and B 


4 


10 


14 


A only 


6 


8 


14 


B only 


8 


5 


13 


Neither 


7 


6 


13 




25 


29 


54 



X2 



= 3.35 



P <.34 



Class 
standing 
Feedback 
(A) 
gi van 



Table 6b - Feedback vs. Satisfaction 
Satisfaction 



^ (1) = 





No 


Yes 




No 


15 


11 


26 


Yes 


10 


18 


28 


25 


2y 


54 



P <.17 



Table 6c - Feedback vs. Satisfaction 
Satisfaction 



Predicted grac'e 
Feedback (Bj 
given 





No 


Yes 




! iNo 

1 

1 


13 


14 


27 


1 Yes 


12 


15 


27 




25 


29 


54 


x2,„ = .00 




P<'.95 





C onclusions 

The analysis indicates that there were no significant differences in 
final exam performance and in satisfaction between the groups. The group 
receiving feedback class standing does appear to be superior on both there 
variables but the small sample used in this experiment would require large 
differences ir order to reach a significance level of^= .05 

There is a signis-icant linear relationship between test grades and 
final exai\ grede, therefore justifying the use of the program to predict 
final grades. The linear regression accounted for 37% of the variance in 
final exam results. 



Discussion 

The students were asked for written comments about the feedback during 
the final examination. An apparent misunderstanding of the nature of feed- 
back type B (predicted final mark) was evident. This was difficult to avoid 
because of the necessity of providing four different modes of feedback to 
a single class and attempting to avoid "leakage' between groups. This mode 
of feedback was earlier found by the author to be extremely popular with second- 
year Engineering students when the results for the whole class were regularly 
posted and the :nethod of obtaining th3 feedback was explained. 

The comments indicated a favorable response to feedback mode A (class 
standing), but more specific information such as rank or standard score based 
on class mean and standard deviation would have been more satisfactory, 
tha choice of quarters (not quartiles) as an indicator of class standing was 
made in order to minimize "leakage" between groups, i.e., a student could 
not determine his rank by using infor.nation obtained from another student. 

It should be noted that the subjects were in their final semester of 
university and other factors such as concern w^'th employment may have reduced 
the impact of the feedback. Perhaps the regularity and not simply the mode 
of the feedback was the important factor for all the groups. Even under the 
restricted conditions of the experiment, the results showed a slight advantage 
for the feedback groups. It is hypothesized that a similar experiment per- 
formed with several separate classes of lower year students would yield mere 
definite results. 

It would be useful to determine whether a higher degree relationship 
exists between test grades and final exam grade, e.g. .quadratic, which would 
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account for a higher proportion of the total variance. If such a relation- 
ship exists, the FORTRAN predicting program should be modified accordingly. 

Although the different feedback modes caused no change in performance, 
the predicting program can effectively be used to monitor the progress of 
students in a course. The "predicted final grade" would aid the instructor 
in identifying potential failures in time to provide the required individual 
assistance. /-v^ 



